2026-07-18 10:28:46,763 [ 716092 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:42, check_args_and_update_paths) 2026-07-18 10:28:46,763 [ 716092 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:86, check_args_and_update_paths) 2026-07-18 10:28:46,764 [ 716092 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:97, check_args_and_update_paths) 2026-07-18 10:28:46,764 [ 716092 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:99, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_5pchpt --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=b74d853ebe76 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS=" -rfEps --run-id=2 --color=no --durations=0 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-8]' -vvv" altinityinfra/integration-tests-runner:cf95077f4609 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: xdist-3.5.0, random-0.2, order-1.0.1, repeat-0.9.4, reportlog-0.4.0, timeout-2.2.0 timeout: 900.0s timeout method: signal timeout func_only: False collecting ... collected 1 item test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] FAILED [100%] =================================== FAILURES =================================== ______________________ test_shards_distributed[ordered-8] ______________________ started_cluster = mode = 'ordered', processing_threads = 8 @pytest.mark.parametrize( "mode, processing_threads", [ pytest.param("unordered", 1), pytest.param("unordered", 8), pytest.param("ordered", 1), pytest.param("ordered", 8), ], ) def test_shards_distributed(started_cluster, mode, processing_threads): node = started_cluster.instances["instance"] node_2 = started_cluster.instances["instance2"] table_name = f"test_shards_distributed_{mode}_{processing_threads}" dst_table_name = f"{table_name}_dst" keeper_path = f"/clickhouse/test_{table_name}" files_path = f"{table_name}_data" files_to_generate = 300 row_num = 50 total_rows = row_num * files_to_generate shards_num = 2 i = 0 for instance in [node, node_2]: create_table( started_cluster, instance, table_name, mode, files_path, additional_settings={ "keeper_path": keeper_path, "s3queue_processing_threads_num": processing_threads, "s3queue_buckets": shards_num, }, ) i += 1 for instance in [node, node_2]: create_mv(instance, table_name, dst_table_name) total_values = generate_random_files( started_cluster, files_path, files_to_generate, row_num=row_num ) def get_count(node, table_name): return int(run_query(node, f"SELECT count() FROM {table_name}")) def print_debug_info(): processed_files = ( node.query( f""" select splitByChar('/', file_name)[-1] as file from system.s3queue where zookeeper_path ilike '%{table_name}%' and status = 'Processed' and rows_processed > 0 order by file """ ) .strip() .split("\n") ) logging.debug( f"Processed files by node 1: {len(processed_files)}/{files_to_generate}" ) processed_files = ( node_2.query( f""" select splitByChar('/', file_name)[-1] as file from system.s3queue where zookeeper_path ilike '%{table_name}%' and status = 'Processed' and rows_processed > 0 order by file """ ) .strip() .split("\n") ) logging.debug( f"Processed files by node 2: {len(processed_files)}/{files_to_generate}" ) count = get_count(node, dst_table_name) + get_count(node_2, dst_table_name) logging.debug(f"Processed rows: {count}/{total_rows}") info = node.query( f""" select concat('test_', toString(number), '.csv') as file from numbers(300) where file not in (select splitByChar('/', file_name)[-1] from clusterAllReplicas(default, system.s3queue) where zookeeper_path ilike '%{table_name}%' and status = 'Processed' and rows_processed > 0) """ ) logging.debug(f"Unprocessed files: {info}") files1 = ( node.query( f""" select splitByChar('/', file_name)[-1] from system.s3queue where zookeeper_path ilike '%{table_name}%' and status = 'Processed' and rows_processed > 0 """ ) .strip() .split("\n") ) files2 = ( node_2.query( f""" select splitByChar('/', file_name)[-1] from system.s3queue where zookeeper_path ilike '%{table_name}%' and status = 'Processed' and rows_processed > 0 """ ) .strip() .split("\n") ) def intersection(list_a, list_b): return [e for e in list_a if e in list_b] logging.debug(f"Intersecting files: {intersection(files1, files2)}") for _ in range(30): if ( get_count(node, dst_table_name) + get_count(node_2, dst_table_name) ) == total_rows: break time.sleep(1) if ( get_count(node, dst_table_name) + get_count(node_2, dst_table_name) ) != total_rows: print_debug_info() assert False get_query = f"SELECT column1, column2, column3 FROM {dst_table_name}" res1 = [list(map(int, l.split())) for l in run_query(node, get_query).splitlines()] res2 = [ list(map(int, l.split())) for l in run_query(node_2, get_query).splitlines() ] if len(res1) + len(res2) != total_rows or len(res1) <= 0 or len(res2) <= 0 or True: logging.debug( f"res1 size: {len(res1)}, res2 size: {len(res2)}, total_rows: {total_rows}" ) print_debug_info() assert len(res1) + len(res2) == total_rows # Checking that all engines have made progress assert len(res1) > 0 > assert len(res2) > 0 E assert 0 > 0 E + where 0 = len([]) test_storage_s3_queue/test.py:1468: AssertionError ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ------------------------------ Captured log setup ------------------------------ 2026-07-18 08:28:49 [ 514 ] DEBUG : Command:['docker ps | wc -l'] (cluster.py:113, run_and_check) 2026-07-18 08:28:49 [ 514 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-07-18 08:28:49 [ 514 ] DEBUG : No running containers (conftest.py:95, cleanup_environment) 2026-07-18 08:28:49 [ 514 ] DEBUG : Pruning Docker networks (conftest.py:97, cleanup_environment) 2026-07-18 08:28:49 [ 514 ] DEBUG : Command:['docker network prune --force'] (cluster.py:113, run_and_check) 2026-07-18 08:28:49 [ 514 ] DEBUG : Command:["sysctl net.ipv4.ip_local_port_range='55000 65535'"] (cluster.py:113, run_and_check) 2026-07-18 08:28:49 [ 514 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:121, run_and_check) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_KERBEROS_KDC_TAG 9391ecdee8d7 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV CLICKHOUSE_TESTS_SERVER_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV MSAN_OPTIONS abort_on_error=1 poison_in_dtor=1 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV JAVA_TOOL_OPTIONS -Djdk.attach.allowAttachSelf=true (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV TSAN_OPTIONS halt_on_error=1 abort_on_error=1 history_size=7 memory_limit_mb=46080 second_deadlock_stack=1 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV HOSTNAME f829a0566d2b (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV SHLVL 0 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV HOME /root (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV OLDPWD / (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_HELPER_TAG 2cffe1eae894 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV PYTHONUNBUFFERED 1 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_PYTHON_BOTTLE_TAG caad4729259e (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV UBSAN_OPTIONS print_stacktrace=1 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV PYTEST_ADDOPTS -rfEps --run-id=2 --color=no --durations=0 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-8]' -vvv (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV CLICKHOUSE_LIBRARY_BRIDGE_BINARY_PATH /clickhouse-library-bridge (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV COMPOSE_HTTP_TIMEOUT 600 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_MYSQL_PHP_CLIENT_TAG 88be89c1e3b6 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_DOTNET_CLIENT_TAG 11de0b29a15d (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV CLICKHOUSE_TESTS_CLIENT_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_MYSQL_JS_CLIENT_TAG 41ba7c2ec2a1 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV PATH /spark-3.3.2-bin-hadoop3/bin:/opt/gdb/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_KERBERIZED_HADOOP_TAG latest (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_CHANNEL stable (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_CLIENT_TIMEOUT 300 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_POSTGRESQL_JAVA_CLIENT_TAG a4eff5c7f4d6 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_NGINX_DAV_TAG b55ac9cd7519 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_MYSQL_GOLANG_CLIENT_TAG 9bec2a638e6e (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV PWD /ClickHouse/tests/integration (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_MYSQL_JAVA_CLIENT_TAG 766bff31cfe4 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV CLICKHOUSE_ODBC_BRIDGE_BINARY_PATH /clickhouse-odbc-bridge (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV CLICKHOUSE_TESTS_BASE_CONFIG_DIR /clickhouse-config (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV TZ Etc/UTC (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV JAVA_PATH /usr/lib/jvm/java-11-openjdk-amd64/bin/java (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV DOCKER_BASE_TAG b74d853ebe76 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV SPARK_HOME /spark-3.3.2-bin-hadoop3 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV LC_CTYPE C.UTF-8 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV INTEGRATION_TESTS_RUN_ID 2 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV WORKER_FREE_PORTS 30000 30001 30002 30003 30004 30005 30006 30007 30008 30009 30010 30011 30012 30013 30014 30015 30016 30017 30018 30019 30020 30021 30022 30023 30024 30025 30026 30027 30028 30029 30030 30031 30032 30033 30034 30035 30036 30037 30038 30039 30040 30041 30042 30043 30044 30045 30046 30047 30048 30049 (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : ENV PYTEST_CURRENT_TEST test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] (setup) (cluster.py:446, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : CLUSTER INIT base_config_dir:/clickhouse-config (cluster.py:766, __init__) 2026-07-18 08:28:49 [ 514 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup Keeper (cluster.py:1059, setup_keeper_cmd) 2026-07-18 08:28:49 [ 514 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:instance tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-07-18 08:28:49 [ 514 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-07-18 08:28:49 [ 514 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:instance2 tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-07-18 08:28:49 [ 514 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-07-18 08:28:49 [ 514 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:old_instance tag:23.12 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-07-18 08:28:49 [ 514 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-07-18 08:28:49 [ 514 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:instance_too_many_parts tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-07-18 08:28:49 [ 514 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-07-18 08:28:49 [ 514 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:instance_24.5 tag:24.5 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-07-18 08:28:49 [ 514 ] INFO : Starting cluster... (test.py:143, started_cluster) 2026-07-18 08:28:49 [ 514 ] INFO : Running tests in /ClickHouse/tests/integration/test_storage_s3_queue/test.py (cluster.py:2790, start) 2026-07-18 08:28:49 [ 514 ] DEBUG : Cluster start called. is_up=False (cluster.py:2797, start) 2026-07-18 08:28:49 [ 514 ] DEBUG : Docker networks for project rootteststorages3queue are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-07-18 08:28:49 [ 514 ] DEBUG : Docker containers for project rootteststorages3queue are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-07-18 08:28:49 [ 514 ] DEBUG : Docker volumes for project rootteststorages3queue are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-07-18 08:28:49 [ 514 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-07-18 08:28:49 [ 514 ] DEBUG : Docker networks for project rootteststorages3queue are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-07-18 08:28:49 [ 514 ] DEBUG : Docker containers for project rootteststorages3queue are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-07-18 08:28:49 [ 514 ] DEBUG : Docker volumes for project rootteststorages3queue are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-07-18 08:28:49 [ 514 ] DEBUG : Command:docker container list --all --filter name='^/rootteststorages3queue_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-07-18 08:28:49 [ 514 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-07-18 08:28:49 [ 514 ] DEBUG : No running containers for project: rootteststorages3queue (cluster.py:904, cleanup) 2026-07-18 08:28:49 [ 514 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-07-18 08:28:49 [ 514 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-07-18 08:28:49 [ 514 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-07-18 08:28:49 [ 514 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup directory for instance: instance (cluster.py:2810, start) 2026-07-18 08:28:49 [ 514 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_storage_s3_queue/configs/zookeeper.xml', '/ClickHouse/tests/integration/test_storage_s3_queue/configs/s3queue_log.xml'] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/configs/config.d (cluster.py:4636, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/database (cluster.py:4653, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/logs (cluster.py:4664, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup directory for instance: instance2 (cluster.py:2810, start) 2026-07-18 08:28:49 [ 514 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_storage_s3_queue/configs/s3queue_log.xml'] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/configs/config.d (cluster.py:4636, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/database (cluster.py:4653, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/logs (cluster.py:4664, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup directory for instance: old_instance (cluster.py:2810, start) 2026-07-18 08:28:49 [ 514 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Copy custom test config files [] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/configs/config.d (cluster.py:4636, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/database (cluster.py:4653, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/logs (cluster.py:4664, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup directory for instance: instance_too_many_parts (cluster.py:2810, start) 2026-07-18 08:28:49 [ 514 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_storage_s3_queue/configs/s3queue_log.xml', '/ClickHouse/tests/integration/test_storage_s3_queue/configs/merge_tree.xml'] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/configs/config.d (cluster.py:4636, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/database (cluster.py:4653, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/logs (cluster.py:4664, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup directory for instance: instance_24.5 (cluster.py:2810, start) 2026-07-18 08:28:49 [ 514 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Copy custom test config files [] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/configs/config.d (cluster.py:4636, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/database (cluster.py:4653, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/logs (cluster.py:4664, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-07-18 08:28:49 [ 514 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:b74d853ebe76', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/coordination', 'MINIO_CERTS_DIR': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/minio/certs', 'MINIO_DATA_DIR': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/minio/data', 'MINIO_PORT': '9001', 'SSL_CERT_FILE': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/minio/certs/public.crt', 'AZURITE_PORT': '30000', 'AZURITE_STORAGE_ACCOUNT_URL': 'http://azurite1:30000/devstoreaccount1', 'AZURITE_CONNECTION_STRING': 'DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:30000/devstoreaccount1;'} stored in /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env (cluster.py:86, _create_env_file) 2026-07-18 08:28:49 [ 514 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-07-18 08:28:49 [ 514 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-07-18 08:28:49 [ 514 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-07-18 08:28:49 [ 514 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-07-18 08:28:49 [ 514 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 825 (connectionpool.py:547, _make_request) 2026-07-18 08:28:49 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo2 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling azurite1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo3 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance_24.5 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling old_instance ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance2 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance_too_many_parts ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance_too_many_parts ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo2 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance_too_many_parts ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance_too_many_parts ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance_too_many_parts ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling minio1 ... pulling from minio/minio (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling minio1 ... digest: sha256:9eea83a4e1425067e7... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling minio1 ... status: image is up to date for m... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling minio1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance_24.5 ... pulling from clickhouse/clickhous... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance_24.5 ... digest: sha256:f1155aa4adec32964c... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance_24.5 ... status: image is up to date for c... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance_24.5 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo3 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:92e86c3de6b8ff00d8... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo3 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling zoo3 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling azurite1 ... pulling from altinityinfra/azurite (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling azurite1 ... digest: sha256:dae2a5f96553962901... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling azurite1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling azurite1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling old_instance ... pulling from clickhouse/clickhous... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:92e86c3de6b8ff00d8... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling old_instance ... digest: sha256:5d0802b64c09f6d2c6... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling old_instance ... status: image is up to date for c... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling old_instance ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Stderr:Pulling instance2 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:51 [ 514 ] DEBUG : Setup ZooKeeper (cluster.py:2853, start) 2026-07-18 08:28:51 [ 514 ] DEBUG : Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/log', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/config', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/log', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/config', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/coordination', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/log', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/config', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/coordination'] (cluster.py:2854, start) 2026-07-18 08:28:51 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'amd64', 'BuildTime': '2023-05-05T21:18:13.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-185-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v2.2.6', 'Details': {'GitCommit': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}}, {'Name': 'runc', 'Version': '1.3.6', 'Details': {'GitCommit': 'v1.3.6-0-g491b69ba'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=amd64, KernelVersion=5.15.0-185-generic, BuildTime=2023-05-05T21:18:13.000000000+00:00 (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'x86_64', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '11ce9d5f3c68c941867e82890e93e815c1304f1b', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'ID': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Containers': 0, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.network.ensure: Creating network "rootteststorages3queue_default" with the default driver (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network <- (name='rootteststorages3queue_default', driver=None, options=None, ipam=None, internal=False, enable_ipv6=False, labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.network': 'default', 'com.docker.compose.version': '1.29.2'}, attachable=True, check_duplicate=True) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network -> {'Id': '87414582a8b8ae2d30b9700e264e85fcda9ba21ec43a158be8aa93e4d302f130', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Warning': ''} (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, , } (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='zoo2', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='zoo2', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='zoo1', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='zoo1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 19a72f805a4e8cb60dda21570cdfd8ec41108468ba9bb233eeebf8e0711b987f (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/log', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Source': '/clickhouse', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Target': '/usr/bin/clickhouse', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config2.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:b74d853ebe76', user='0', volumes={}, name='rootteststorages3queue_zoo2_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'zoo2', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '19a72f805a4e8cb60dda21570cdfd8ec41108468ba9bb233eeebf8e0711b987f'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/log', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['zoo2'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo3_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='zoo3', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='zoo3', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 644d03b791809042b81650137d0ec21069457d4dcd0c57c975ab1be967be7b93 (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/config', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Source': '/clickhouse', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Target': '/usr/bin/clickhouse', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config1.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:b74d853ebe76', user='0', volumes={}, name='rootteststorages3queue_zoo1_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'zoo1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '644d03b791809042b81650137d0ec21069457d4dcd0c57c975ab1be967be7b93'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/config', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['zoo1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: de92082730bd2f35caa2348f0ccac93081b6c7b0b2644c000b1716432e303676 (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/log', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Source': '/clickhouse', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Target': '/usr/bin/clickhouse', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config3.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:b74d853ebe76', user='0', volumes={}, name='rootteststorages3queue_zoo3_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'zoo3', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'de92082730bd2f35caa2348f0ccac93081b6c7b0b2644c000b1716432e303676'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/log', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['zoo3'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '31273b619029cb9c63da140effe41cfc567218313d9d927e556890139ad863a5', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('31273b619029cb9c63da140effe41cfc567218313d9d927e556890139ad863a5') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('31273b619029cb9c63da140effe41cfc567218313d9d927e556890139ad863a5', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '1d912a1d1afd8c947af946ea87284d1fd7ee7203eb057e786c0d936f80a851eb', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('1d912a1d1afd8c947af946ea87284d1fd7ee7203eb057e786c0d936f80a851eb') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '0d36e8fab4e2b684335bda8728850e362d9f43e7d91220092d948d41ec6891cd', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('0d36e8fab4e2b684335bda8728850e362d9f43e7d91220092d948d41ec6891cd') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('0d36e8fab4e2b684335bda8728850e362d9f43e7d91220092d948d41ec6891cd', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('1d912a1d1afd8c947af946ea87284d1fd7ee7203eb057e786c0d936f80a851eb', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('31273b619029cb9c63da140effe41cfc567218313d9d927e556890139ad863a5', 'rootteststorages3queue_default', aliases=['zoo2', '31273b619029'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('0d36e8fab4e2b684335bda8728850e362d9f43e7d91220092d948d41ec6891cd', 'rootteststorages3queue_default', aliases=['0d36e8fab4e2', 'zoo1'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('1d912a1d1afd8c947af946ea87284d1fd7ee7203eb057e786c0d936f80a851eb', 'rootteststorages3queue_default', aliases=['1d912a1d1afd', 'zoo3'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('31273b619029cb9c63da140effe41cfc567218313d9d927e556890139ad863a5') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('0d36e8fab4e2b684335bda8728850e362d9f43e7d91220092d948d41ec6891cd') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('1d912a1d1afd8c947af946ea87284d1fd7ee7203eb057e786c0d936f80a851eb') (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='zoo2', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='zoo3', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='zoo1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:52 [ 514 ] DEBUG : Wait ZooKeeper to start (cluster.py:2506, wait_zookeeper_to_start) 2026-07-18 08:28:52 [ 514 ] DEBUG : get_instance_ip instance_name=zoo1 (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:52 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_zoo1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:52 [ 514 ] DEBUG : get_kazoo_client: zoo1, ip:172.16.1.3, port:2181, use_ssl:False (cluster.py:3290, get_kazoo_client) 2026-07-18 08:28:52 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:52 [ 514 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-07-18 08:28:52 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:52 [ 514 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-07-18 08:28:53 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:53 [ 514 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-07-18 08:28:53 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:53 [ 514 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-07-18 08:28:54 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:54 [ 514 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-07-18 08:28:56 [ 514 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:56 [ 514 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-07-18 08:28:56 [ 514 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-07-18 08:28:56 [ 514 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-07-18 08:28:56 [ 514 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-07-18 08:28:56 [ 514 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-07-18 08:28:56 [ 514 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-07-18 08:28:56 [ 514 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-07-18 08:28:56 [ 514 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-07-18 08:28:56 [ 514 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-07-18 08:28:56 [ 514 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-07-18 08:28:56 [ 514 ] DEBUG : get_instance_ip instance_name=zoo2 (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:56 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_zoo2_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:56 [ 514 ] DEBUG : get_kazoo_client: zoo2, ip:172.16.1.2, port:2181, use_ssl:False (cluster.py:3290, get_kazoo_client) 2026-07-18 08:28:56 [ 514 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:56 [ 514 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-07-18 08:28:56 [ 514 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-07-18 08:28:56 [ 514 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-07-18 08:28:56 [ 514 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-07-18 08:28:56 [ 514 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-07-18 08:28:56 [ 514 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-07-18 08:28:56 [ 514 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-07-18 08:28:56 [ 514 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-07-18 08:28:56 [ 514 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-07-18 08:28:56 [ 514 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-07-18 08:28:56 [ 514 ] DEBUG : get_instance_ip instance_name=zoo3 (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:56 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_zoo3_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:56 [ 514 ] DEBUG : get_kazoo_client: zoo3, ip:172.16.1.4, port:2181, use_ssl:False (cluster.py:3290, get_kazoo_client) 2026-07-18 08:28:56 [ 514 ] INFO : Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False (connection.py:650, _connect) 2026-07-18 08:28:56 [ 514 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-07-18 08:28:56 [ 514 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-07-18 08:28:56 [ 514 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-07-18 08:28:56 [ 514 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-07-18 08:28:56 [ 514 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-07-18 08:28:56 [ 514 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-07-18 08:28:56 [ 514 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-07-18 08:28:56 [ 514 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-07-18 08:28:56 [ 514 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-07-18 08:28:56 [ 514 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-07-18 08:28:56 [ 514 ] DEBUG : All instances of ZooKeeper Secure started (cluster.py:2521, wait_zookeeper_nodes_to_start) 2026-07-18 08:28:56 [ 514 ] INFO : Trying to create Minio instance by command docker-compose --env-file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env --project-name rootteststorages3queue --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --verbose up -d (cluster.py:3091, start) 2026-07-18 08:28:56 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'amd64', 'BuildTime': '2023-05-05T21:18:13.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-185-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v2.2.6', 'Details': {'GitCommit': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}}, {'Name': 'runc', 'Version': '1.3.6', 'Details': {'GitCommit': 'v1.3.6-0-g491b69ba'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=amd64, KernelVersion=5.15.0-185-generic, BuildTime=2023-05-05T21:18:13.000000000+00:00 (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Containers': {'0d36e8fab4e2b684335bda8728850e362d9f43e7d91220092d948d41ec6891cd': {'EndpointID': 'e5719f3976736f578f840ba584a0ce5a68459388d4235345bb48f4a61de1e5df', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.3/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:01:03', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_zoo1_1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '1d912a1d1afd8c947af946ea87284d1fd7ee7203eb057e786c0d936f80a851eb': {'EndpointID': '08e7a2f2487d0acc54b49386545d6e282498fddc3602727549688f23d1e07213', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.4/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'x86_64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '11ce9d5f3c68c941867e82890e93e815c1304f1b', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'ID': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Containers': 3, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Containers': {'0d36e8fab4e2b684335bda8728850e362d9f43e7d91220092d948d41ec6891cd': {'EndpointID': 'e5719f3976736f578f840ba584a0ce5a68459388d4235345bb48f4a61de1e5df', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.3/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:01:03', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_zoo1_1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '1d912a1d1afd8c947af946ea87284d1fd7ee7203eb057e786c0d936f80a851eb': {'EndpointID': '08e7a2f2487d0acc54b49386545d6e282498fddc3602727549688f23d1e07213', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.4/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.volume.initialize: Creating volume "rootteststorages3queue_data1-1" with default driver (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_volume <- ('rootteststorages3queue_data1-1', None, None, labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.volume': 'data1-1', 'com.docker.compose.version': '1.29.2'}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_volume -> {'CreatedAt': '2026-07-18T08:28:56Z', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_data1-1/_data', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_data1-1', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 3 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 3 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('0d36e8fab4e2b684335bda8728850e362d9f43e7d91220092d948d41ec6891cd') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('31273b619029cb9c63da140effe41cfc567218313d9d927e556890139ad863a5') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('1d912a1d1afd8c947af946ea87284d1fd7ee7203eb057e786c0d936f80a851eb') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.project.find_orphan_containers: Found orphan containers (rootteststorages3queue_zoo1_1, rootteststorages3queue_zoo2_1, rootteststorages3queue_zoo3_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('minio/minio:RELEASE.2024-07-31T05-46-26Z') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['minio'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/usr/bin/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/python-bottle:caad4729259e') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.project._get_convergence_plans: minio1 has upstream changes (proxy1, proxy2) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.project._get_convergence_plans: resolver has upstream changes (proxy1, proxy2) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, , , } (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_proxy1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='proxy1', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='proxy1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_proxy2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='proxy2', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='proxy2', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: b56c6836595b51d9a20583ad32fc4f8944a1765818910b4dd803f05a12e9f3e4 (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (image='altinityinfra/s3-proxy', volumes={}, name='rootteststorages3queue_proxy1_1', detach=True, ports=['8080', '80', '443'], environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'proxy1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'b56c6836595b51d9a20583ad32fc4f8944a1765818910b4dd803f05a12e9f3e4'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['proxy1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: b56c6836595b51d9a20583ad32fc4f8944a1765818910b4dd803f05a12e9f3e4 (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (image='altinityinfra/s3-proxy', volumes={}, name='rootteststorages3queue_proxy2_1', detach=True, ports=['8080', '80', '443'], environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'proxy2', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'b56c6836595b51d9a20583ad32fc4f8944a1765818910b4dd803f05a12e9f3e4'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['proxy2'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '25f3c1ddeceece0b4724fc7d8e8fba6ce561b9347b14098995e78123e7a6a67a', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('25f3c1ddeceece0b4724fc7d8e8fba6ce561b9347b14098995e78123e7a6a67a') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': 'cabd258752facff444ecf8b1a7fb0b1bcacb26c913d0b44687176ae868ce5cbb', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('cabd258752facff444ecf8b1a7fb0b1bcacb26c913d0b44687176ae868ce5cbb') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('25f3c1ddeceece0b4724fc7d8e8fba6ce561b9347b14098995e78123e7a6a67a', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('cabd258752facff444ecf8b1a7fb0b1bcacb26c913d0b44687176ae868ce5cbb', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('cabd258752facff444ecf8b1a7fb0b1bcacb26c913d0b44687176ae868ce5cbb', 'rootteststorages3queue_default', aliases=['cabd258752fa', 'proxy2'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('25f3c1ddeceece0b4724fc7d8e8fba6ce561b9347b14098995e78123e7a6a67a', 'rootteststorages3queue_default', aliases=['proxy1', '25f3c1ddecee'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('cabd258752facff444ecf8b1a7fb0b1bcacb26c913d0b44687176ae868ce5cbb') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('25f3c1ddeceece0b4724fc7d8e8fba6ce561b9347b14098995e78123e7a6a67a') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='proxy1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_proxy1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='proxy2', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_proxy2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_minio1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='minio1', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='minio1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_resolver_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='resolver', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='resolver', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('minio/minio:RELEASE.2024-07-31T05-46-26Z') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/python-bottle:caad4729259e') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['minio'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/usr/bin/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('minio/minio:RELEASE.2024-07-31T05-46-26Z') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['minio'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/usr/bin/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/python-bottle:caad4729259e') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: d259040ce9b11414a99758790da684e95a32b4972eacbfbab4dea5a65344ddf5 (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=['rootteststorages3queue_data1-1:/data1:rw', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/minio/certs:/certs:rw'], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': ['rootteststorages3queue_data1-1:/data1:rw', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/minio/certs:/certs:rw'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (command='server --console-address 127.0.0.1:19001 --address :9001 --certs-dir /certs /data1-1', environment=['MINIO_ACCESS_KEY=minio', 'MINIO_SECRET_KEY=minio123', 'MINIO_PROMETHEUS_AUTH_TYPE=public'], image='minio/minio:RELEASE.2024-07-31T05-46-26Z', volumes={'/data1': {}, '/certs': {}}, name='rootteststorages3queue_minio1_1', detach=True, ports=['9001'], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'minio1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'd259040ce9b11414a99758790da684e95a32b4972eacbfbab4dea5a65344ddf5'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': ['rootteststorages3queue_data1-1:/data1:rw', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/minio/certs:/certs:rw'], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['minio1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 6c39b51bdd1ee6641e47438265edd23f4a911da06c476fb341022c357974cf87 (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (image='altinityinfra/python-bottle:caad4729259e', tty=True, volumes={}, name='rootteststorages3queue_resolver_1', detach=True, ports=['8080'], environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'resolver', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '6c39b51bdd1ee6641e47438265edd23f4a911da06c476fb341022c357974cf87'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['resolver'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '3825ce654d71e45f372c0cf514e39b8e27fde565d609fed6de48ab4297789978', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('3825ce654d71e45f372c0cf514e39b8e27fde565d609fed6de48ab4297789978') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Args': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': 'dbd7652df309423bf8e5c3c6a11e646f9f0ab9f645e19b3ae5a7a7338b3a8811', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('dbd7652df309423bf8e5c3c6a11e646f9f0ab9f645e19b3ae5a7a7338b3a8811') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('3825ce654d71e45f372c0cf514e39b8e27fde565d609fed6de48ab4297789978', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Args': ['server', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '--console-address', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '127.0.0.1:19001', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '--address', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: ':9001', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '--certs-dir', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '/certs', (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: '/data1-1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('dbd7652df309423bf8e5c3c6a11e646f9f0ab9f645e19b3ae5a7a7338b3a8811', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('3825ce654d71e45f372c0cf514e39b8e27fde565d609fed6de48ab4297789978', 'rootteststorages3queue_default', aliases=['3825ce654d71', 'resolver'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('dbd7652df309423bf8e5c3c6a11e646f9f0ab9f645e19b3ae5a7a7338b3a8811', 'rootteststorages3queue_default', aliases=['dbd7652df309', 'minio1'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('3825ce654d71e45f372c0cf514e39b8e27fde565d609fed6de48ab4297789978') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('dbd7652df309423bf8e5c3c6a11e646f9f0ab9f645e19b3ae5a7a7338b3a8811') (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='minio1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_minio1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='resolver', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_resolver_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:57 [ 514 ] INFO : Trying to connect to Minio... (cluster.py:3097, start) 2026-07-18 08:28:57 [ 514 ] DEBUG : get_instance_ip instance_name=minio1 (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:57 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_minio1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:57 [ 514 ] DEBUG : get_instance_ip instance_name=proxy1 (cluster.py:2137, get_instance_ip) 2026-07-18 08:28:57 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_proxy1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:28:57 [ 514 ] DEBUG : Starting new HTTP connection (1): 172.16.1.8:9001 (connectionpool.py:245, _new_conn) 2026-07-18 08:28:57 [ 514 ] DEBUG : http://172.16.1.8:9001 "GET / HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:57 [ 514 ] DEBUG : Connected to Minio. (cluster.py:2651, wait_minio_to_start) 2026-07-18 08:28:57 [ 514 ] DEBUG : http://172.16.1.8:9001 "GET /root?location= HTTP/1.1" 404 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:57 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:57 [ 514 ] DEBUG : S3 bucket 'root' created (cluster.py:2666, wait_minio_to_start) 2026-07-18 08:28:57 [ 514 ] DEBUG : http://172.16.1.8:9001 "GET /root2?location= HTTP/1.1" 404 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:57 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root2 HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:57 [ 514 ] DEBUG : S3 bucket 'root2' created (cluster.py:2666, wait_minio_to_start) 2026-07-18 08:28:57 [ 514 ] INFO : Trying to create Azurite instance by command docker-compose --env-file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env --project-name rootteststorages3queue --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml --verbose up -d (cluster.py:3102, start) 2026-07-18 08:28:57 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'amd64', 'BuildTime': '2023-05-05T21:18:13.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-185-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v2.2.6', 'Details': {'GitCommit': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}}, {'Name': 'runc', 'Version': '1.3.6', 'Details': {'GitCommit': 'v1.3.6-0-g491b69ba'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=amd64, KernelVersion=5.15.0-185-generic, BuildTime=2023-05-05T21:18:13.000000000+00:00 (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Containers': {'0d36e8fab4e2b684335bda8728850e362d9f43e7d91220092d948d41ec6891cd': {'EndpointID': 'e5719f3976736f578f840ba584a0ce5a68459388d4235345bb48f4a61de1e5df', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.3/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:01:03', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_zoo1_1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '1d912a1d1afd8c947af946ea87284d1fd7ee7203eb057e786c0d936f80a851eb': {'EndpointID': '08e7a2f2487d0acc54b49386545d6e282498fddc3602727549688f23d1e07213', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.4/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume -> {'CreatedAt': '2026-07-18T08:28:56Z', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_data1-1/_data', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_data1-1', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'x86_64', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '11ce9d5f3c68c941867e82890e93e815c1304f1b', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'ID': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Containers': 7, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Containers': {'0d36e8fab4e2b684335bda8728850e362d9f43e7d91220092d948d41ec6891cd': {'EndpointID': 'e5719f3976736f578f840ba584a0ce5a68459388d4235345bb48f4a61de1e5df', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.3/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:01:03', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_zoo1_1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '1d912a1d1afd8c947af946ea87284d1fd7ee7203eb057e786c0d936f80a851eb': {'EndpointID': '08e7a2f2487d0acc54b49386545d6e282498fddc3602727549688f23d1e07213', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.4/24', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume -> {'CreatedAt': '2026-07-18T08:28:56Z', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_data1-1/_data', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_data1-1', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume -> {'CreatedAt': '2026-07-18T08:28:56Z', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_data1-1/_data', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_data1-1', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 7 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 7 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('0d36e8fab4e2b684335bda8728850e362d9f43e7d91220092d948d41ec6891cd') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('25f3c1ddeceece0b4724fc7d8e8fba6ce561b9347b14098995e78123e7a6a67a') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('31273b619029cb9c63da140effe41cfc567218313d9d927e556890139ad863a5') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('cabd258752facff444ecf8b1a7fb0b1bcacb26c913d0b44687176ae868ce5cbb') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('1d912a1d1afd8c947af946ea87284d1fd7ee7203eb057e786c0d936f80a851eb') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('3825ce654d71e45f372c0cf514e39b8e27fde565d609fed6de48ab4297789978') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Args': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('dbd7652df309423bf8e5c3c6a11e646f9f0ab9f645e19b3ae5a7a7338b3a8811') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Args': ['server', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--console-address', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '127.0.0.1:19001', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--address', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: ':9001', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--certs-dir', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '/certs', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '/data1-1'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.project.find_orphan_containers: Found orphan containers (rootteststorages3queue_zoo1_1, rootteststorages3queue_proxy1_1, rootteststorages3queue_zoo2_1, rootteststorages3queue_proxy2_1, rootteststorages3queue_zoo3_1, rootteststorages3queue_resolver_1, rootteststorages3queue_minio1_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/azurite:3.35.0') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Cmd': ['azurite', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '-l', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '/data', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {} (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_azurite1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='azurite1', number=1)} (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='azurite1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/azurite:3.35.0') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Cmd': ['azurite', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '-l', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '/data', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/azurite:3.35.0') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Cmd': ['azurite', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '-l', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '/data', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 2ece5e07b89b0f4e7b5e8076ad1e1b0717274882ab654de56a4c7613ded50eb6 (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={'30000/tcp': ['30000']}, binds=['rootteststorages3queue_data1-1:/data1:rw'], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': ['rootteststorages3queue_data1-1:/data1:rw'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'PortBindings': {'30000/tcp': [{'HostIp': '', 'HostPort': '30000'}]}, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (command='azurite-blob --blobHost 0.0.0.0 --blobPort 30000 --debug /azurite_log', image='altinityinfra/azurite:3.35.0', ports=[('30000', 'tcp')], volumes={'/data1': {}}, name='rootteststorages3queue_azurite1_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'azurite1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_azurite.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '2ece5e07b89b0f4e7b5e8076ad1e1b0717274882ab654de56a4c7613ded50eb6'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': ['rootteststorages3queue_data1-1:/data1:rw'], 'PortBindings': {'30000/tcp': [{'HostIp': '', 'HostPort': '30000'}]}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['azurite1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': 'ed615e4f587dfeac01fecefbd08e7ef834bc969809fe33200ee07900e9e2d4eb', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('ed615e4f587dfeac01fecefbd08e7ef834bc969809fe33200ee07900e9e2d4eb') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Args': ['azurite-blob', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--blobHost', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '0.0.0.0', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--blobPort', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '30000', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '--debug', (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: '/azurite_log'], (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('ed615e4f587dfeac01fecefbd08e7ef834bc969809fe33200ee07900e9e2d4eb', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('ed615e4f587dfeac01fecefbd08e7ef834bc969809fe33200ee07900e9e2d4eb', 'rootteststorages3queue_default', aliases=['ed615e4f587d', 'azurite1'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('ed615e4f587dfeac01fecefbd08e7ef834bc969809fe33200ee07900e9e2d4eb') (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='azurite1', number=1) (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_azurite1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-07-18 08:28:58 [ 514 ] INFO : Trying to connect to Azurite (cluster.py:3118, start) 2026-07-18 08:28:59 [ 514 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-185-generic-x86_64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': 'b973846a-8282-11f1-bc9a-a609304d895f' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-07-18 08:28:59 [ 514 ] DEBUG : Starting new HTTP connection (1): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-07-18 08:28:59 [ 514 ] DEBUG : http://127.0.0.1:30000 "GET /devstoreaccount1/?restype=account&comp=properties HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:28:59 [ 514 ] INFO : Response status: 200 Response headers: 'Server': 'Azurite-Blob/3.35.0' 'x-ms-client-request-id': 'b973846a-8282-11f1-bc9a-a609304d895f' 'x-ms-request-id': '70fe5460-b754-4ee3-aa57-e75f6f601a9d' 'x-ms-version': 'REDACTED' 'date': 'Sat, 18 Jul 2026 08:28:59 GMT' 'x-ms-sku-name': 'REDACTED' 'x-ms-account-kind': 'REDACTED' 'x-ms-is-hns-enabled': 'REDACTED' 'Connection': 'keep-alive' 'Keep-Alive': 'REDACTED' 'Content-Length': '0' (_universal.py:549, on_response) 2026-07-18 08:28:59 [ 514 ] DEBUG : {'client_request_id': 'b973846a-8282-11f1-bc9a-a609304d895f', 'request_id': '70fe5460-b754-4ee3-aa57-e75f6f601a9d', 'version': '2025-11-05', 'date': datetime.datetime(2026, 7, 18, 8, 28, 59, tzinfo=datetime.timezone.utc), 'sku_name': 'Standard_RAGRS', 'account_kind': 'StorageV2', 'is_hns_enabled': False} (cluster.py:2699, wait_azurite_to_start) 2026-07-18 08:28:59 [ 514 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker-compose --env-file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env --project-name rootteststorages3queue --file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml --file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml --file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml --file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml --file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/docker-compose.yml up -d --no-recreate') (cluster.py:3150, start) 2026-07-18 08:28:59 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/docker-compose.yml', 'up', '-d', '--no-recreate'] (cluster.py:113, run_and_check) 2026-07-18 08:29:00 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_old_instance_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:00 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance_24.5_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:00 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:00 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance_too_many_parts_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:00 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:00 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:00 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance_too_many_parts_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:00 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance_24.5_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:00 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_old_instance_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:00 [ 514 ] DEBUG : Stderr:Creating rootteststorages3queue_instance2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:00 [ 514 ] DEBUG : ClickHouse instance created (cluster.py:3158, start) 2026-07-18 08:29:00 [ 514 ] DEBUG : get_instance_ip instance_name=instance (cluster.py:2137, get_instance_ip) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : Waiting for ClickHouse start in instance, ip: 172.16.1.10... (cluster.py:3165, start) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/6df7de268d53e5f61d44205645d3f9e8d860fe8d9853b7b6e96be03fe80e4c36/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : ClickHouse instance started (cluster.py:3169, start) 2026-07-18 08:29:00 [ 514 ] DEBUG : get_instance_ip instance_name=instance2 (cluster.py:2137, get_instance_ip) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance2_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : Waiting for ClickHouse start in instance2, ip: 172.16.1.14... (cluster.py:3165, start) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance2_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/f72e95b46335d7a46f971498223a5e0c9893b4f2f584830c78342679a55f7f7e/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/f72e95b46335d7a46f971498223a5e0c9893b4f2f584830c78342679a55f7f7e/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/f72e95b46335d7a46f971498223a5e0c9893b4f2f584830c78342679a55f7f7e/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : ClickHouse instance2 started (cluster.py:3169, start) 2026-07-18 08:29:00 [ 514 ] DEBUG : get_instance_ip instance_name=old_instance (cluster.py:2137, get_instance_ip) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_old_instance_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : Waiting for ClickHouse start in old_instance, ip: 172.16.1.11... (cluster.py:3165, start) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_old_instance_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/d9fe2763360155502b992d6c2c768f59f9ecf09b0b1926b88be03af8750b3413/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/d9fe2763360155502b992d6c2c768f59f9ecf09b0b1926b88be03af8750b3413/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : ClickHouse old_instance started (cluster.py:3169, start) 2026-07-18 08:29:00 [ 514 ] DEBUG : get_instance_ip instance_name=instance_too_many_parts (cluster.py:2137, get_instance_ip) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance_too_many_parts_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : Waiting for ClickHouse start in instance_too_many_parts, ip: 172.16.1.12... (cluster.py:3165, start) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance_too_many_parts_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/104c8ddff5567b4eba6fa55fb4cd92b4d5c39857cec223d7710eedd191262c1e/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : ClickHouse instance_too_many_parts started (cluster.py:3169, start) 2026-07-18 08:29:00 [ 514 ] DEBUG : get_instance_ip instance_name=instance_24.5 (cluster.py:2137, get_instance_ip) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance_24.5_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : Waiting for ClickHouse start in instance_24.5, ip: 172.16.1.13... (cluster.py:3165, start) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_instance_24.5_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://localhost:None "GET /v1.42/containers/b0c534dfce7773bb4cf9ac3b8155a595a2d666b04cc61a04e159226d060d4675/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : ClickHouse instance_24.5 started (cluster.py:3169, start) 2026-07-18 08:29:00 [ 514 ] INFO : Cluster started (test.py:145, started_cluster) 2026-07-18 08:29:00 [ 514 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/cont?restype=REDACTED' Request method: 'PUT' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-185-generic-x86_64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': 'ba6086d4-8282-11f1-bc9a-a609304d895f' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-07-18 08:29:00 [ 514 ] DEBUG : http://127.0.0.1:30000 "PUT /devstoreaccount1/cont?restype=container HTTP/1.1" 201 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:00 [ 514 ] INFO : Response status: 201 Response headers: 'Server': 'Azurite-Blob/3.35.0' 'etag': '"0x1E4FD97DBE77970"' 'last-modified': 'Sat, 18 Jul 2026 08:29:00 GMT' 'x-ms-client-request-id': 'ba6086d4-8282-11f1-bc9a-a609304d895f' 'x-ms-request-id': '1636959b-5b41-42f8-b357-d4ef74ce9108' 'x-ms-version': 'REDACTED' 'Date': 'Sat, 18 Jul 2026 08:29:00 GMT' 'Connection': 'keep-alive' 'Keep-Alive': 'REDACTED' 'Content-Length': '0' (_universal.py:549, on_response) 2026-07-18 08:29:00 [ 514 ] DEBUG : Executing query DROP DATABASE IF EXISTS test; CREATE DATABASE test; on instance (cluster.py:3606, query) 2026-07-18 08:29:00 [ 514 ] DEBUG : Executing query DROP DATABASE IF EXISTS test; CREATE DATABASE test; on instance2 (cluster.py:3606, query) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "GET /root?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix= HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) ----------------------------- Captured stdout call ----------------------------- Generating files: [('test_shards_distributed_ordered_8_data/test_0.csv', 0), ('test_shards_distributed_ordered_8_data/test_1.csv', 1), ('test_shards_distributed_ordered_8_data/test_10.csv', 10), ('test_shards_distributed_ordered_8_data/test_100.csv', 100), ('test_shards_distributed_ordered_8_data/test_101.csv', 101), ('test_shards_distributed_ordered_8_data/test_102.csv', 102), ('test_shards_distributed_ordered_8_data/test_103.csv', 103), ('test_shards_distributed_ordered_8_data/test_104.csv', 104), ('test_shards_distributed_ordered_8_data/test_105.csv', 105), ('test_shards_distributed_ordered_8_data/test_106.csv', 106), ('test_shards_distributed_ordered_8_data/test_107.csv', 107), ('test_shards_distributed_ordered_8_data/test_108.csv', 108), ('test_shards_distributed_ordered_8_data/test_109.csv', 109), ('test_shards_distributed_ordered_8_data/test_11.csv', 11), ('test_shards_distributed_ordered_8_data/test_110.csv', 110), ('test_shards_distributed_ordered_8_data/test_111.csv', 111), ('test_shards_distributed_ordered_8_data/test_112.csv', 112), ('test_shards_distributed_ordered_8_data/test_113.csv', 113), ('test_shards_distributed_ordered_8_data/test_114.csv', 114), ('test_shards_distributed_ordered_8_data/test_115.csv', 115), ('test_shards_distributed_ordered_8_data/test_116.csv', 116), ('test_shards_distributed_ordered_8_data/test_117.csv', 117), ('test_shards_distributed_ordered_8_data/test_118.csv', 118), ('test_shards_distributed_ordered_8_data/test_119.csv', 119), ('test_shards_distributed_ordered_8_data/test_12.csv', 12), ('test_shards_distributed_ordered_8_data/test_120.csv', 120), ('test_shards_distributed_ordered_8_data/test_121.csv', 121), ('test_shards_distributed_ordered_8_data/test_122.csv', 122), ('test_shards_distributed_ordered_8_data/test_123.csv', 123), ('test_shards_distributed_ordered_8_data/test_124.csv', 124), ('test_shards_distributed_ordered_8_data/test_125.csv', 125), ('test_shards_distributed_ordered_8_data/test_126.csv', 126), ('test_shards_distributed_ordered_8_data/test_127.csv', 127), ('test_shards_distributed_ordered_8_data/test_128.csv', 128), ('test_shards_distributed_ordered_8_data/test_129.csv', 129), ('test_shards_distributed_ordered_8_data/test_13.csv', 13), ('test_shards_distributed_ordered_8_data/test_130.csv', 130), ('test_shards_distributed_ordered_8_data/test_131.csv', 131), ('test_shards_distributed_ordered_8_data/test_132.csv', 132), ('test_shards_distributed_ordered_8_data/test_133.csv', 133), ('test_shards_distributed_ordered_8_data/test_134.csv', 134), ('test_shards_distributed_ordered_8_data/test_135.csv', 135), ('test_shards_distributed_ordered_8_data/test_136.csv', 136), ('test_shards_distributed_ordered_8_data/test_137.csv', 137), ('test_shards_distributed_ordered_8_data/test_138.csv', 138), ('test_shards_distributed_ordered_8_data/test_139.csv', 139), ('test_shards_distributed_ordered_8_data/test_14.csv', 14), ('test_shards_distributed_ordered_8_data/test_140.csv', 140), ('test_shards_distributed_ordered_8_data/test_141.csv', 141), ('test_shards_distributed_ordered_8_data/test_142.csv', 142), ('test_shards_distributed_ordered_8_data/test_143.csv', 143), ('test_shards_distributed_ordered_8_data/test_144.csv', 144), ('test_shards_distributed_ordered_8_data/test_145.csv', 145), ('test_shards_distributed_ordered_8_data/test_146.csv', 146), ('test_shards_distributed_ordered_8_data/test_147.csv', 147), ('test_shards_distributed_ordered_8_data/test_148.csv', 148), ('test_shards_distributed_ordered_8_data/test_149.csv', 149), ('test_shards_distributed_ordered_8_data/test_15.csv', 15), ('test_shards_distributed_ordered_8_data/test_150.csv', 150), ('test_shards_distributed_ordered_8_data/test_151.csv', 151), ('test_shards_distributed_ordered_8_data/test_152.csv', 152), ('test_shards_distributed_ordered_8_data/test_153.csv', 153), ('test_shards_distributed_ordered_8_data/test_154.csv', 154), ('test_shards_distributed_ordered_8_data/test_155.csv', 155), ('test_shards_distributed_ordered_8_data/test_156.csv', 156), ('test_shards_distributed_ordered_8_data/test_157.csv', 157), ('test_shards_distributed_ordered_8_data/test_158.csv', 158), ('test_shards_distributed_ordered_8_data/test_159.csv', 159), ('test_shards_distributed_ordered_8_data/test_16.csv', 16), ('test_shards_distributed_ordered_8_data/test_160.csv', 160), ('test_shards_distributed_ordered_8_data/test_161.csv', 161), ('test_shards_distributed_ordered_8_data/test_162.csv', 162), ('test_shards_distributed_ordered_8_data/test_163.csv', 163), ('test_shards_distributed_ordered_8_data/test_164.csv', 164), ('test_shards_distributed_ordered_8_data/test_165.csv', 165), ('test_shards_distributed_ordered_8_data/test_166.csv', 166), ('test_shards_distributed_ordered_8_data/test_167.csv', 167), ('test_shards_distributed_ordered_8_data/test_168.csv', 168), ('test_shards_distributed_ordered_8_data/test_169.csv', 169), ('test_shards_distributed_ordered_8_data/test_17.csv', 17), ('test_shards_distributed_ordered_8_data/test_170.csv', 170), ('test_shards_distributed_ordered_8_data/test_171.csv', 171), ('test_shards_distributed_ordered_8_data/test_172.csv', 172), ('test_shards_distributed_ordered_8_data/test_173.csv', 173), ('test_shards_distributed_ordered_8_data/test_174.csv', 174), ('test_shards_distributed_ordered_8_data/test_175.csv', 175), ('test_shards_distributed_ordered_8_data/test_176.csv', 176), ('test_shards_distributed_ordered_8_data/test_177.csv', 177), ('test_shards_distributed_ordered_8_data/test_178.csv', 178), ('test_shards_distributed_ordered_8_data/test_179.csv', 179), ('test_shards_distributed_ordered_8_data/test_18.csv', 18), ('test_shards_distributed_ordered_8_data/test_180.csv', 180), ('test_shards_distributed_ordered_8_data/test_181.csv', 181), ('test_shards_distributed_ordered_8_data/test_182.csv', 182), ('test_shards_distributed_ordered_8_data/test_183.csv', 183), ('test_shards_distributed_ordered_8_data/test_184.csv', 184), ('test_shards_distributed_ordered_8_data/test_185.csv', 185), ('test_shards_distributed_ordered_8_data/test_186.csv', 186), ('test_shards_distributed_ordered_8_data/test_187.csv', 187), ('test_shards_distributed_ordered_8_data/test_188.csv', 188), ('test_shards_distributed_ordered_8_data/test_189.csv', 189), ('test_shards_distributed_ordered_8_data/test_19.csv', 19), ('test_shards_distributed_ordered_8_data/test_190.csv', 190), ('test_shards_distributed_ordered_8_data/test_191.csv', 191), ('test_shards_distributed_ordered_8_data/test_192.csv', 192), ('test_shards_distributed_ordered_8_data/test_193.csv', 193), ('test_shards_distributed_ordered_8_data/test_194.csv', 194), ('test_shards_distributed_ordered_8_data/test_195.csv', 195), ('test_shards_distributed_ordered_8_data/test_196.csv', 196), ('test_shards_distributed_ordered_8_data/test_197.csv', 197), ('test_shards_distributed_ordered_8_data/test_198.csv', 198), ('test_shards_distributed_ordered_8_data/test_199.csv', 199), ('test_shards_distributed_ordered_8_data/test_2.csv', 2), ('test_shards_distributed_ordered_8_data/test_20.csv', 20), ('test_shards_distributed_ordered_8_data/test_200.csv', 200), ('test_shards_distributed_ordered_8_data/test_201.csv', 201), ('test_shards_distributed_ordered_8_data/test_202.csv', 202), ('test_shards_distributed_ordered_8_data/test_203.csv', 203), ('test_shards_distributed_ordered_8_data/test_204.csv', 204), ('test_shards_distributed_ordered_8_data/test_205.csv', 205), ('test_shards_distributed_ordered_8_data/test_206.csv', 206), ('test_shards_distributed_ordered_8_data/test_207.csv', 207), ('test_shards_distributed_ordered_8_data/test_208.csv', 208), ('test_shards_distributed_ordered_8_data/test_209.csv', 209), ('test_shards_distributed_ordered_8_data/test_21.csv', 21), ('test_shards_distributed_ordered_8_data/test_210.csv', 210), ('test_shards_distributed_ordered_8_data/test_211.csv', 211), ('test_shards_distributed_ordered_8_data/test_212.csv', 212), ('test_shards_distributed_ordered_8_data/test_213.csv', 213), ('test_shards_distributed_ordered_8_data/test_214.csv', 214), ('test_shards_distributed_ordered_8_data/test_215.csv', 215), ('test_shards_distributed_ordered_8_data/test_216.csv', 216), ('test_shards_distributed_ordered_8_data/test_217.csv', 217), ('test_shards_distributed_ordered_8_data/test_218.csv', 218), ('test_shards_distributed_ordered_8_data/test_219.csv', 219), ('test_shards_distributed_ordered_8_data/test_22.csv', 22), ('test_shards_distributed_ordered_8_data/test_220.csv', 220), ('test_shards_distributed_ordered_8_data/test_221.csv', 221), ('test_shards_distributed_ordered_8_data/test_222.csv', 222), ('test_shards_distributed_ordered_8_data/test_223.csv', 223), ('test_shards_distributed_ordered_8_data/test_224.csv', 224), ('test_shards_distributed_ordered_8_data/test_225.csv', 225), ('test_shards_distributed_ordered_8_data/test_226.csv', 226), ('test_shards_distributed_ordered_8_data/test_227.csv', 227), ('test_shards_distributed_ordered_8_data/test_228.csv', 228), ('test_shards_distributed_ordered_8_data/test_229.csv', 229), ('test_shards_distributed_ordered_8_data/test_23.csv', 23), ('test_shards_distributed_ordered_8_data/test_230.csv', 230), ('test_shards_distributed_ordered_8_data/test_231.csv', 231), ('test_shards_distributed_ordered_8_data/test_232.csv', 232), ('test_shards_distributed_ordered_8_data/test_233.csv', 233), ('test_shards_distributed_ordered_8_data/test_234.csv', 234), ('test_shards_distributed_ordered_8_data/test_235.csv', 235), ('test_shards_distributed_ordered_8_data/test_236.csv', 236), ('test_shards_distributed_ordered_8_data/test_237.csv', 237), ('test_shards_distributed_ordered_8_data/test_238.csv', 238), ('test_shards_distributed_ordered_8_data/test_239.csv', 239), ('test_shards_distributed_ordered_8_data/test_24.csv', 24), ('test_shards_distributed_ordered_8_data/test_240.csv', 240), ('test_shards_distributed_ordered_8_data/test_241.csv', 241), ('test_shards_distributed_ordered_8_data/test_242.csv', 242), ('test_shards_distributed_ordered_8_data/test_243.csv', 243), ('test_shards_distributed_ordered_8_data/test_244.csv', 244), ('test_shards_distributed_ordered_8_data/test_245.csv', 245), ('test_shards_distributed_ordered_8_data/test_246.csv', 246), ('test_shards_distributed_ordered_8_data/test_247.csv', 247), ('test_shards_distributed_ordered_8_data/test_248.csv', 248), ('test_shards_distributed_ordered_8_data/test_249.csv', 249), ('test_shards_distributed_ordered_8_data/test_25.csv', 25), ('test_shards_distributed_ordered_8_data/test_250.csv', 250), ('test_shards_distributed_ordered_8_data/test_251.csv', 251), ('test_shards_distributed_ordered_8_data/test_252.csv', 252), ('test_shards_distributed_ordered_8_data/test_253.csv', 253), ('test_shards_distributed_ordered_8_data/test_254.csv', 254), ('test_shards_distributed_ordered_8_data/test_255.csv', 255), ('test_shards_distributed_ordered_8_data/test_256.csv', 256), ('test_shards_distributed_ordered_8_data/test_257.csv', 257), ('test_shards_distributed_ordered_8_data/test_258.csv', 258), ('test_shards_distributed_ordered_8_data/test_259.csv', 259), ('test_shards_distributed_ordered_8_data/test_26.csv', 26), ('test_shards_distributed_ordered_8_data/test_260.csv', 260), ('test_shards_distributed_ordered_8_data/test_261.csv', 261), ('test_shards_distributed_ordered_8_data/test_262.csv', 262), ('test_shards_distributed_ordered_8_data/test_263.csv', 263), ('test_shards_distributed_ordered_8_data/test_264.csv', 264), ('test_shards_distributed_ordered_8_data/test_265.csv', 265), ('test_shards_distributed_ordered_8_data/test_266.csv', 266), ('test_shards_distributed_ordered_8_data/test_267.csv', 267), ('test_shards_distributed_ordered_8_data/test_268.csv', 268), ('test_shards_distributed_ordered_8_data/test_269.csv', 269), ('test_shards_distributed_ordered_8_data/test_27.csv', 27), ('test_shards_distributed_ordered_8_data/test_270.csv', 270), ('test_shards_distributed_ordered_8_data/test_271.csv', 271), ('test_shards_distributed_ordered_8_data/test_272.csv', 272), ('test_shards_distributed_ordered_8_data/test_273.csv', 273), ('test_shards_distributed_ordered_8_data/test_274.csv', 274), ('test_shards_distributed_ordered_8_data/test_275.csv', 275), ('test_shards_distributed_ordered_8_data/test_276.csv', 276), ('test_shards_distributed_ordered_8_data/test_277.csv', 277), ('test_shards_distributed_ordered_8_data/test_278.csv', 278), ('test_shards_distributed_ordered_8_data/test_279.csv', 279), ('test_shards_distributed_ordered_8_data/test_28.csv', 28), ('test_shards_distributed_ordered_8_data/test_280.csv', 280), ('test_shards_distributed_ordered_8_data/test_281.csv', 281), ('test_shards_distributed_ordered_8_data/test_282.csv', 282), ('test_shards_distributed_ordered_8_data/test_283.csv', 283), ('test_shards_distributed_ordered_8_data/test_284.csv', 284), ('test_shards_distributed_ordered_8_data/test_285.csv', 285), ('test_shards_distributed_ordered_8_data/test_286.csv', 286), ('test_shards_distributed_ordered_8_data/test_287.csv', 287), ('test_shards_distributed_ordered_8_data/test_288.csv', 288), ('test_shards_distributed_ordered_8_data/test_289.csv', 289), ('test_shards_distributed_ordered_8_data/test_29.csv', 29), ('test_shards_distributed_ordered_8_data/test_290.csv', 290), ('test_shards_distributed_ordered_8_data/test_291.csv', 291), ('test_shards_distributed_ordered_8_data/test_292.csv', 292), ('test_shards_distributed_ordered_8_data/test_293.csv', 293), ('test_shards_distributed_ordered_8_data/test_294.csv', 294), ('test_shards_distributed_ordered_8_data/test_295.csv', 295), ('test_shards_distributed_ordered_8_data/test_296.csv', 296), ('test_shards_distributed_ordered_8_data/test_297.csv', 297), ('test_shards_distributed_ordered_8_data/test_298.csv', 298), ('test_shards_distributed_ordered_8_data/test_299.csv', 299), ('test_shards_distributed_ordered_8_data/test_3.csv', 3), ('test_shards_distributed_ordered_8_data/test_30.csv', 30), ('test_shards_distributed_ordered_8_data/test_31.csv', 31), ('test_shards_distributed_ordered_8_data/test_32.csv', 32), ('test_shards_distributed_ordered_8_data/test_33.csv', 33), ('test_shards_distributed_ordered_8_data/test_34.csv', 34), ('test_shards_distributed_ordered_8_data/test_35.csv', 35), ('test_shards_distributed_ordered_8_data/test_36.csv', 36), ('test_shards_distributed_ordered_8_data/test_37.csv', 37), ('test_shards_distributed_ordered_8_data/test_38.csv', 38), ('test_shards_distributed_ordered_8_data/test_39.csv', 39), ('test_shards_distributed_ordered_8_data/test_4.csv', 4), ('test_shards_distributed_ordered_8_data/test_40.csv', 40), ('test_shards_distributed_ordered_8_data/test_41.csv', 41), ('test_shards_distributed_ordered_8_data/test_42.csv', 42), ('test_shards_distributed_ordered_8_data/test_43.csv', 43), ('test_shards_distributed_ordered_8_data/test_44.csv', 44), ('test_shards_distributed_ordered_8_data/test_45.csv', 45), ('test_shards_distributed_ordered_8_data/test_46.csv', 46), ('test_shards_distributed_ordered_8_data/test_47.csv', 47), ('test_shards_distributed_ordered_8_data/test_48.csv', 48), ('test_shards_distributed_ordered_8_data/test_49.csv', 49), ('test_shards_distributed_ordered_8_data/test_5.csv', 5), ('test_shards_distributed_ordered_8_data/test_50.csv', 50), ('test_shards_distributed_ordered_8_data/test_51.csv', 51), ('test_shards_distributed_ordered_8_data/test_52.csv', 52), ('test_shards_distributed_ordered_8_data/test_53.csv', 53), ('test_shards_distributed_ordered_8_data/test_54.csv', 54), ('test_shards_distributed_ordered_8_data/test_55.csv', 55), ('test_shards_distributed_ordered_8_data/test_56.csv', 56), ('test_shards_distributed_ordered_8_data/test_57.csv', 57), ('test_shards_distributed_ordered_8_data/test_58.csv', 58), ('test_shards_distributed_ordered_8_data/test_59.csv', 59), ('test_shards_distributed_ordered_8_data/test_6.csv', 6), ('test_shards_distributed_ordered_8_data/test_60.csv', 60), ('test_shards_distributed_ordered_8_data/test_61.csv', 61), ('test_shards_distributed_ordered_8_data/test_62.csv', 62), ('test_shards_distributed_ordered_8_data/test_63.csv', 63), ('test_shards_distributed_ordered_8_data/test_64.csv', 64), ('test_shards_distributed_ordered_8_data/test_65.csv', 65), ('test_shards_distributed_ordered_8_data/test_66.csv', 66), ('test_shards_distributed_ordered_8_data/test_67.csv', 67), ('test_shards_distributed_ordered_8_data/test_68.csv', 68), ('test_shards_distributed_ordered_8_data/test_69.csv', 69), ('test_shards_distributed_ordered_8_data/test_7.csv', 7), ('test_shards_distributed_ordered_8_data/test_70.csv', 70), ('test_shards_distributed_ordered_8_data/test_71.csv', 71), ('test_shards_distributed_ordered_8_data/test_72.csv', 72), ('test_shards_distributed_ordered_8_data/test_73.csv', 73), ('test_shards_distributed_ordered_8_data/test_74.csv', 74), ('test_shards_distributed_ordered_8_data/test_75.csv', 75), ('test_shards_distributed_ordered_8_data/test_76.csv', 76), ('test_shards_distributed_ordered_8_data/test_77.csv', 77), ('test_shards_distributed_ordered_8_data/test_78.csv', 78), ('test_shards_distributed_ordered_8_data/test_79.csv', 79), ('test_shards_distributed_ordered_8_data/test_8.csv', 8), ('test_shards_distributed_ordered_8_data/test_80.csv', 80), ('test_shards_distributed_ordered_8_data/test_81.csv', 81), ('test_shards_distributed_ordered_8_data/test_82.csv', 82), ('test_shards_distributed_ordered_8_data/test_83.csv', 83), ('test_shards_distributed_ordered_8_data/test_84.csv', 84), ('test_shards_distributed_ordered_8_data/test_85.csv', 85), ('test_shards_distributed_ordered_8_data/test_86.csv', 86), ('test_shards_distributed_ordered_8_data/test_87.csv', 87), ('test_shards_distributed_ordered_8_data/test_88.csv', 88), ('test_shards_distributed_ordered_8_data/test_89.csv', 89), ('test_shards_distributed_ordered_8_data/test_9.csv', 9), ('test_shards_distributed_ordered_8_data/test_90.csv', 90), ('test_shards_distributed_ordered_8_data/test_91.csv', 91), ('test_shards_distributed_ordered_8_data/test_92.csv', 92), ('test_shards_distributed_ordered_8_data/test_93.csv', 93), ('test_shards_distributed_ordered_8_data/test_94.csv', 94), ('test_shards_distributed_ordered_8_data/test_95.csv', 95), ('test_shards_distributed_ordered_8_data/test_96.csv', 96), ('test_shards_distributed_ordered_8_data/test_97.csv', 97), ('test_shards_distributed_ordered_8_data/test_98.csv', 98), ('test_shards_distributed_ordered_8_data/test_99.csv', 99)] ------------------------------ Captured log call ------------------------------- 2026-07-18 08:29:01 [ 514 ] DEBUG : Executing query DROP TABLE IF EXISTS test_shards_distributed_ordered_8 on instance (cluster.py:3606, query) 2026-07-18 08:29:01 [ 514 ] DEBUG : Executing query CREATE TABLE test_shards_distributed_ordered_8 (column1 UInt32, column2 UInt32, column3 UInt32) ENGINE = S3Queue('http://minio1:9001/root/test_shards_distributed_ordered_8_data/', 'minio','minio123', CSV) SETTINGS s3queue_loading_retries=0,after_processing='keep',keeper_path='/clickhouse/test_test_shards_distributed_ordered_8',mode='ordered',s3queue_processing_threads_num=8,s3queue_buckets=2 on instance (cluster.py:3606, query) 2026-07-18 08:29:01 [ 514 ] DEBUG : Executing query DROP TABLE IF EXISTS test_shards_distributed_ordered_8 on instance2 (cluster.py:3606, query) 2026-07-18 08:29:01 [ 514 ] DEBUG : Executing query CREATE TABLE test_shards_distributed_ordered_8 (column1 UInt32, column2 UInt32, column3 UInt32) ENGINE = S3Queue('http://minio1:9001/root/test_shards_distributed_ordered_8_data/', 'minio','minio123', CSV) SETTINGS s3queue_loading_retries=0,after_processing='keep',keeper_path='/clickhouse/test_test_shards_distributed_ordered_8',mode='ordered',s3queue_processing_threads_num=8,s3queue_buckets=2 on instance2 (cluster.py:3606, query) 2026-07-18 08:29:01 [ 514 ] DEBUG : Executing query DROP TABLE IF EXISTS test_shards_distributed_ordered_8_dst; DROP TABLE IF EXISTS test_shards_distributed_ordered_8_dst_mv; CREATE TABLE test_shards_distributed_ordered_8_dst (column1 UInt32, column2 UInt32, column3 UInt32, _path String) ENGINE = MergeTree() ORDER BY column1; CREATE MATERIALIZED VIEW test_shards_distributed_ordered_8_dst_mv TO test_shards_distributed_ordered_8_dst AS SELECT *, _path FROM test_shards_distributed_ordered_8; on instance (cluster.py:3606, query) 2026-07-18 08:29:01 [ 514 ] DEBUG : Executing query DROP TABLE IF EXISTS test_shards_distributed_ordered_8_dst; DROP TABLE IF EXISTS test_shards_distributed_ordered_8_dst_mv; CREATE TABLE test_shards_distributed_ordered_8_dst (column1 UInt32, column2 UInt32, column3 UInt32, _path String) ENGINE = MergeTree() ORDER BY column1; CREATE MATERIALIZED VIEW test_shards_distributed_ordered_8_dst_mv TO test_shards_distributed_ordered_8_dst AS SELECT *, _path FROM test_shards_distributed_ordered_8; on instance2 (cluster.py:3606, query) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_0.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_1.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_10.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_100.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_101.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_102.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_103.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_104.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_105.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_106.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_107.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_108.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_109.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_11.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_110.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_111.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_112.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_113.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_114.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_115.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_116.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_117.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_118.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_119.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_12.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_120.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_121.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_122.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_123.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_124.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_125.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_126.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_127.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_128.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_129.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_13.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_130.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_131.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_132.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_133.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_134.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_135.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_136.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_137.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_138.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_139.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_14.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_140.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_141.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_142.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_143.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_144.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_145.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_146.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_147.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_148.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_149.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_15.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_150.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_151.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_152.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_153.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_154.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_155.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_156.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_157.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_158.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_159.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_16.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_160.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_161.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_162.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_163.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_164.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_165.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_166.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_167.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_168.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_169.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_17.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_170.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_171.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_172.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_173.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_174.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_175.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_176.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_177.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_178.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_179.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_18.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_180.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_181.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_182.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_183.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_184.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_185.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_186.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_187.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_188.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_189.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_19.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_190.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_191.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_192.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_193.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_194.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_195.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_196.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_197.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_198.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_199.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_2.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_20.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_200.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_201.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_202.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_203.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_204.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_205.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_206.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_207.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_208.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_209.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_21.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_210.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_211.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_212.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_213.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_214.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_215.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_216.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_217.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_218.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_219.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_22.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_220.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_221.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_222.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_223.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_224.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_225.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_226.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_227.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_228.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_229.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_23.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_230.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_231.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_232.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_233.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_234.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_235.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_236.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_237.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_238.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_239.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_24.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_240.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_241.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_242.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_243.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_244.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_245.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_246.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_247.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_248.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_249.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_25.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_250.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_251.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_252.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_253.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_254.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_255.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_256.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_257.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_258.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_259.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_26.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_260.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_261.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_262.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_263.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_264.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_265.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_266.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_267.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_268.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_269.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_27.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_270.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_271.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_272.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_273.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_274.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_275.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_276.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_277.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_278.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_279.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_28.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_280.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_281.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_282.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_283.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_284.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_285.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_286.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_287.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_288.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_289.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_29.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_290.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_291.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_292.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_293.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_294.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_295.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_296.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_297.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_298.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_299.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_3.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_30.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_31.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_32.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_33.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_34.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_35.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_36.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_37.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_38.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_39.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_4.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_40.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_41.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_42.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_43.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_44.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_45.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_46.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_47.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_48.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_49.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_5.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_50.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_51.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_52.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_53.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_54.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_55.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_56.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_57.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_58.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_59.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_6.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_60.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_61.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_62.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_63.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_64.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_65.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_66.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_67.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_68.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_69.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_7.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_70.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_71.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_72.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_73.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_74.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_75.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_76.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_77.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_78.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_79.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_8.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_80.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_81.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_82.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_83.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_84.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_85.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_86.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_87.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_88.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_89.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_9.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_90.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_91.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_92.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_93.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_94.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_95.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:01 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_96.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:02 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_97.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:02 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_98.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:02 [ 514 ] DEBUG : http://172.16.1.8:9001 "PUT /root/test_shards_distributed_ordered_8_data/test_99.csv HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-07-18 08:29:02 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:29:02 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:29:02 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:29:02 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:29:02 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:29:02 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:29:03 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:29:03 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:29:03 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:29:03 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:29:03 [ 514 ] INFO : Running query 'SELECT column1, column2, column3 FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query SELECT column1, column2, column3 FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:29:03 [ 514 ] INFO : Running query 'SELECT column1, column2, column3 FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query SELECT column1, column2, column3 FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:29:03 [ 514 ] DEBUG : res1 size: 15000, res2 size: 0, total_rows: 15000 (test.py:1459, test_shards_distributed) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query select splitByChar('/', file_name)[-1] as file from system.s3queue where zookeeper_path ilike '%test_shards_distributed_ordered_8%' and status = 'Processed' and rows_processed > 0 order by file on instance (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Processed files by node 1: 300/300 (test.py:1384, print_debug_info) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query select splitByChar('/', file_name)[-1] as file from system.s3queue where zookeeper_path ilike '%test_shards_distributed_ordered_8%' and status = 'Processed' and rows_processed > 0 order by file on instance2 (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Processed files by node 2: 1/300 (test.py:1396, print_debug_info) 2026-07-18 08:29:03 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:29:03 [ 514 ] INFO : Running query 'SELECT count() FROM test_shards_distributed_ordered_8_dst'... (test.py:160, run_query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query SELECT count() FROM test_shards_distributed_ordered_8_dst on instance2 (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] INFO : Query finished (test.py:162, run_query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Processed rows: 15000/15000 (test.py:1401, print_debug_info) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query select concat('test_', toString(number), '.csv') as file from numbers(300) where file not in (select splitByChar('/', file_name)[-1] from clusterAllReplicas(default, system.s3queue) where zookeeper_path ilike '%test_shards_distributed_ordered_8%' and status = 'Processed' and rows_processed > 0) on instance (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Unprocessed files: (test.py:1410, print_debug_info) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query select splitByChar('/', file_name)[-1] from system.s3queue where zookeeper_path ilike '%test_shards_distributed_ordered_8%' and status = 'Processed' and rows_processed > 0 on instance (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Executing query select splitByChar('/', file_name)[-1] from system.s3queue where zookeeper_path ilike '%test_shards_distributed_ordered_8%' and status = 'Processed' and rows_processed > 0 on instance2 (cluster.py:3606, query) 2026-07-18 08:29:03 [ 514 ] DEBUG : Intersecting files: [] (test.py:1436, print_debug_info) ---------------------------- Captured log teardown ----------------------------- 2026-07-18 08:29:04 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:113, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance_too_many_parts_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance_24.5_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_old_instance_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_azurite1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_resolver_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_minio1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_proxy2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_proxy1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo3_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_old_instance_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance_too_many_parts_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_instance_24.5_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_minio1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_azurite1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_resolver_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_proxy2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Stopping rootteststorages3queue_proxy1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/docker-compose.yml', 'down', '--volumes'] (cluster.py:113, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance_too_many_parts_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance_24.5_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_old_instance_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_azurite1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_resolver_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_minio1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_proxy2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_proxy1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo1_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo3_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo2_1 ... (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance_too_many_parts_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance_24.5_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_proxy2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_proxy1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_resolver_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_old_instance_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_azurite1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_instance2_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing rootteststorages3queue_minio1_1 ... done (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing network rootteststorages3queue_default (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Stderr:Removing volume rootteststorages3queue_data1-1 (cluster.py:123, run_and_check) 2026-07-18 08:29:25 [ 514 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-07-18 08:29:26 [ 514 ] DEBUG : Docker networks for project rootteststorages3queue are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-07-18 08:29:26 [ 514 ] DEBUG : Docker containers for project rootteststorages3queue are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-07-18 08:29:26 [ 514 ] DEBUG : Docker volumes for project rootteststorages3queue are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-07-18 08:29:26 [ 514 ] DEBUG : Command:docker container list --all --filter name='^/rootteststorages3queue_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-07-18 08:29:26 [ 514 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-07-18 08:29:26 [ 514 ] DEBUG : No running containers for project: rootteststorages3queue (cluster.py:904, cleanup) 2026-07-18 08:29:26 [ 514 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-07-18 08:29:26 [ 514 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-07-18 08:29:26 [ 514 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-07-18 08:29:26 [ 514 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) ============================== slowest durations =============================== 22.03s teardown test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] 11.94s setup test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] 2.97s call test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] =========================== short test summary info ============================ FAILED test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] - as... ============================== 1 failed in 37.00s ============================== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 445, in subprocess.check_call(cmd, shell=True) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_5pchpt --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=b74d853ebe76 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS=" -rfEps --run-id=2 --color=no --durations=0 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-8]' -vvv" altinityinfra/integration-tests-runner:cf95077f4609 ' returned non-zero exit status 1.